An abstract syntax tree (AST) is a data structure used in a compiler to represent the structure of program source code. The AST is an abstract representation of the code that shows associations between nodes in a branching structure. Source code is provided to a parser at compile time, which creates an AST corresponding to the source code. The compiler can then recursively walk the AST and generate machine-readable code. When walking the tree, the compiler needs to understand each node type in order to properly generate an output.
The parser must recognize and understand the keywords used in the source code or it will not know how to create the AST nodes for the keywords. Traditionally, extensions to a programming language required that the keywords be hardcoded and defined in the parser to enable it to build the AST. Additionally, the compiler would have to be modified to handle a new keyword in order to generate executable code. Accordingly, existing parsers cannot handle arbitrary keywords. Instead, the parser and compiler have needed to be revised to add new keywords to a language.